X-SDR: An Extensible Experimentation Suite for Dimensionality Reduction
نویسندگان
چکیده
Due to the vast amount and pace of high-dimensional data production, dimensionality reduction emerges as an important requirement in many application areas. In this paper, we introduce X-SDR, a prototype designed specifically for the deployment and assessment of dimensionality reduction techniques. X-SDR is an integrated environment for dimensionality reduction and knowledge discovery that can be effectively used in the data mining process. In the current version, it supports communication with different database management systems and integrates a wealth of dimensionality reduction algorithms both distributed and centralized. Additionally, it interacts with Weka thus enabling the exploitation of the data mining algorithms therein. Finally, X-SDR provides an API that enables the integration and evaluation of any dimensionality reduction algorithm.
منابع مشابه
A Unified Framework for Dimension Reduction in Forecasting
Factor models are widely used in summarizing large datasets with few underlying latent factors and in building time series forecasting models for economic variables. In these models, the reduction of the predictors and the modeling and forecasting of the response y are carried out in two separate and independent phases. We introduce a potentially more attractive alternative, Sufficient Dimensio...
متن کاملSufficient Dimension Reduction Summaries
Observational studies assessing causal or non-causal relationships between an explanatory measure and an outcome can be complicated by hosts of confounding measures. Large numbers of confounders can lead to several biases in conventional regression based estimation. Inference is more easily conducted if we reduce the number of confounders to a more manageable number. We discuss use of sufficien...
متن کاملKernel Dimension Reduction in Regression∗
We present a new methodology for sufficient dimension reduction (SDR). Our methodology derives directly from the formulation of SDR in terms of the conditional independence of the covariate X from the response Y , given the projection of X on the central subspace [cf. J. Amer. Statist. Assoc. 86 (1991) 316–342 and Regression Graphics (1998) Wiley]. We show that this conditional independence ass...
متن کاملSufficient Dimension Reduction and Modeling Responses Conditioned on Covariates: An Integrated Approach via Convex Optimization
Given observations of a collection of covariates and responses (Y,X) ∈ R × R, sufficient dimension reduction (SDR) techniques aim to identify a mapping f : R → R with k q such that Y |f(X) is independent of X. The image f(X) summarizes the relevant information in a potentially large number of covariates X that influence the responses Y . In many contemporary settings, the number of responses p ...
متن کاملSIR: Dimension Reduction in the Presence of Linearly or Nonlinearly Related Predictors
Sufficient dimension reduction (sdr) is an effective tool for reducing highdimensional predictor spaces in regression problems. sdr achieves dimension reduction without loss of any regression information and without the need to assume any particular parametric form of a model. This is particularly useful for high-dimensional applications such as data mining, marketing, and bioinformatics. Howev...
متن کامل